Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The arguments against open source AI are bad

tombedor.dev

Post reply on HN
Print view View original

31–40 of 228 posts

Re: The arguments against open source AI are bad

#31

Post by TSiege »

nater5000 wrote: post #15

>Much of the angst around China's models centers on "losing the AI race". But what's the goal of this race? Is it to develop the best model? To sell the most tokens? To destroy humanity first? Some people would say it is reaching some sort of singularity. Even if you don't buy into a more sci-fi interpretation of this, there are pretty grounded arguments one could make that there is some sort of "goal" in AI developm…

If AI were to become a super weapon why should I trust a private company to own it? If the super weapon is publicly available to download then why should American citizens be banned from doing so?

Re: The arguments against open source AI are bad

#32

Post by andy99 »

NitpickLawyer wrote: post #23
AlexErrant wrote: post #19

I'm in favor of open source AI... however: > It's theoretically possible for a bad actor to embed hidden adversarial behavior in a model. But if this happens, it serves the interests of responsible actors to find these exploits as soon as possible, and the best way to do this is to let anyone who wants to inspect them. This is a bad argument. It isn't trivial to tell if the weights have poisoned: ``` https://www.thed…

> It isn't trivial to tell if the weights have poisoned: True, but it is easier if you have the weights than if you don't. I guess this was their point?

You should assume the weights have been poisoned, the model has been prompt injected, etc and design software around it accordingly. You can’t really trust the model and it makes sense to always have controls around it and not depend on it behaving a certain way.

I’m aware this doesn’t happen, just saying. There are more examples of a model randomly hallucinating and deleting something than of a deliberate compromise.

Re: The arguments against open source AI are bad

#33

Post by OkayPhysicist »

nater5000 wrote: post #15

>Much of the angst around China's models centers on "losing the AI race". But what's the goal of this race? Is it to develop the best model? To sell the most tokens? To destroy humanity first? Some people would say it is reaching some sort of singularity. Even if you don't buy into a more sci-fi interpretation of this, there are pretty grounded arguments one could make that there is some sort of "goal" in AI developm…

If it's a superweapon, then it's unconstitutional to ban it. Huzzah for the 2nd Ammendment.

Re: The arguments against open source AI are bad

#34

Post by OkayPhysicist »

vouaobrasil wrote: post #9
deaton wrote: post #4

The only good arguments I see against open weight AI also apply to closed AI. And regardless, the box is open, nobody can stop it even if stopping it was a good thing.

It could be stopped, if there were a social movement large enough to make AI a taboo.

Good luck with your Butlerian Jihad.

Re: The arguments against open source AI are bad

#35

Post by esseph »

catigula wrote: post #20
wizzwizz4 wrote: post #14

Earlier quoted context omitted.

I already know how to kill thousands of people. It's not conceptually difficult: more than a day's effort, sure, but what stops people from doing this kind of thing is not the lack of knowing how. https://xkcd.com/1958/

No, you don't. What stops people is actually access to knowledge. There are killers of varying levels of efficacy; making killing easier means more people die. It's 1-1.

> What stops people is actually access to knowledge.

Very untrue.

I'm a combat vet. I spent years fighting an insurgency and therefore pretty good at that very task. I have the knowledge, so what stops me?

Re: The arguments against open source AI are bad

#36

Post by kingstnap »

throw1234567891 wrote: post #29
kingstnap wrote: post #16

Earlier quoted context omitted.

Whats to stop me from going down to the local gas station, filling up a few thousand liters of gas in a rented moving truck, and driving into my nearest hospital? Most people aren't terrorists. You can't just argue stuff is dangerous because it can be one piece of a sophisticated plot.

Nobody will sell you a few thousand litres of gasoline at a gas station.

You don't have to buy it all at once all in one place.

A sophisticated actor can accomplish a lot. You can't bubble wrap the whole world.

The real thing to care about are the incentives and having strong morals and norms. Something that makes me deeply concerned about the US going full mask off recently.

Re: The arguments against open source AI are bad

#37

Post by OkayPhysicist »

catigula wrote: post #8

You could literally develop a hyper-intelligent advisor on how to kill people or perform dangerous hacks using ablated 'open source' AI. You can do this right now, this very moment, and have an extremely adept advisor on how to do really, really bad things.

So? The primary thing stopping people from doing really really bad things is, and always has been, most people not wanting to do bad things.

It's never been difficult. Literally, right this second, you could grab a pen off your desk and pretty easy ram it into the neck of your nearest coworker. Killing's easy. You don't need some hyperintellect to tell you how to do it.

The reason you haven't rammed a pen into your coworker's neck is the same reason why most possible really bad things don't happen: People don't want to do them.

Re: The arguments against open source AI are bad

#38

Post by [deleted] »

AlexErrant wrote: post #19

I'm in favor of open source AI... however: > It's theoretically possible for a bad actor to embed hidden adversarial behavior in a model. But if this happens, it serves the interests of responsible actors to find these exploits as soon as possible, and the best way to do this is to let anyone who wants to inspect them. This is a bad argument. It isn't trivial to tell if the weights have poisoned: ``` https://www.thed…

[deleted]

Re: The arguments against open source AI are bad

#39

Post by CamperBob2 »

AlexErrant wrote: post #19

I'm in favor of open source AI... however: > It's theoretically possible for a bad actor to embed hidden adversarial behavior in a model. But if this happens, it serves the interests of responsible actors to find these exploits as soon as possible, and the best way to do this is to let anyone who wants to inspect them. This is a bad argument. It isn't trivial to tell if the weights have poisoned: ``` https://www.thed…

Exactly. Faced with a choice between open weights that might be poisoned and walled-off weights that I know are poisoned... well, it's an easy choice.
Post reply on HN