Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The arguments against open source AI are bad

tombedor.dev

Post reply on HN
Print view View original

51–60 of 228 posts

Re: The arguments against open source AI are bad

#51

Post by iamnothere »

abernard1 wrote: post #43
_aavaa_ wrote: post #18

Earlier quoted context omitted.

Amazing how we quickly the crypto wars have been forgotten and the lessons not learned.

"But what about criminals?! What about terrorists?! Won't someone please think of the children?!!!" Every argument against open source AI is moot. Outside of totalitarianism, you cannot control people building algorithms with math and software. And even that won't work in the long run. People are not going to stop their AI printing presses because the Church of Venture Capital needs regulatory scarcity to command val…

> Outside of totalitarianism

The flaw in your argument is assuming that decision makers don’t want this

Re: The arguments against open source AI are bad

#52

Post by tomrod »

elmer2 wrote: post #47

We have continued to find backdoored Chinese manufactured routers and network devices. Will there ever be a way to fully audit Chinese models?

Audit them for what, exactly?

Backdoor hardware is straightforward -- its letting people in that the customer doesn't authorize.

Open weights means you aren't tied to any harnessing. So the security domain to be really concerned about is limited to weights only, and I welcome pushback on that.

Are we thinking adversarial injection? Lying about history / propaganda infusion? What is the angle that makes a non-sovereign model dangerous in a way a sovereign model isn't?

Re: The arguments against open source AI are bad

#53

Post by iamnothere »

catigula wrote: post #48
OkayPhysicist wrote: post #37

Earlier quoted context omitted.

So? The primary thing stopping people from doing really really bad things is, and always has been, most people not wanting to do bad things. It's never been difficult. Literally, right this second, you could grab a pen off your desk and pretty easy ram it into the neck of your nearest coworker. Killing's easy. You don't need some hyperintellect to tell you how to do it. The reason you haven't rammed a pen into your c…

It’s interesting that most of the examples people making this argument give are high consequence, high difficulty (could you actually murder your coworker with a pen? I’m skeptical.) low kill count methodologies of violence. We’re specifically talking about low difficulty, highly effective, high kill count methodologies of violence. And yes, being able to kill a lot of people trivially, even with huge consequences is…

> Most countries don’t let you run around with an armory as a result of that.

My country allows it!

Re: The arguments against open source AI are bad

#54

Post by iamnothere »

vouaobrasil wrote: post #9
deaton wrote: post #4

The only good arguments I see against open weight AI also apply to closed AI. And regardless, the box is open, nobody can stop it even if stopping it was a good thing.

It could be stopped, if there were a social movement large enough to make AI a taboo.

Exactly, just like taboos solved racism

Re: The arguments against open source AI are bad

#55

Post by elmer2 »

kittikitti wrote: post #42

They don't have to convince developers, they just have to convince the general population. I've had several discussions about open source with people who don't care about coding and it's hard to untangle the misinformation they receive from the media. They get talking points from authorities that they don't understand so they will always double down on it. For example, rhetoric relating open source to communism is hu…

"For example, rhetoric relating open source to communism is humiliating to discuss"

All Open Source isn't communism. The GPL license is a form of digital communism. It forces you to open source any additions made to open source code as a way to make things 'equal'.

I'm glad it and Stallman are mostly in the dust heap of history and better licenses, like the MIT, BSD, and Apache have gotten popular.

The irony is that the only open source projects surviving long term are funded by very large corporations.

If you don't care about creating code, none of this should really matter.

Re: The arguments against open source AI are bad

#56

Post by bluefirebrand »

ekidd wrote: post #49
TSiege wrote: post #31

Earlier quoted context omitted.

If AI were to become a super weapon why should I trust a private company to own it? If the super weapon is publicly available to download then why should American citizens be banned from doing so?

> If AI were to become a super weapon why should I trust a private company to own it? We have just recently established that: 1. OpenAI's internal "Galaxy" model is fully capable of functioning as what security people refer to as an "Advanced Persistent Threat." The published details of the recent sandbox escape and Hugging Face attack involved chaining multiple unknown zero-days at various stages of the attack, and…

> But yes, if AI becomes an even more dangerous weapon that that, it's time to start asking questions like "What the hell are we doing, anyway

The answer seems to be "getting the weapon before other people get the weapon" which is unfortunate

Re: The arguments against open source AI are bad

#57

Post by efficax »

elmer2 wrote: post #47

We have continued to find backdoored Chinese manufactured routers and network devices. Will there ever be a way to fully audit Chinese models?

will there ever be a way to fully audit american models? hell, at least they're releasing the weights for these so they actually could be audited!

Re: The arguments against open source AI are bad

#58

Post by iamnothere »

bluefirebrand wrote: post #56
ekidd wrote: post #49

Earlier quoted context omitted.

> If AI were to become a super weapon why should I trust a private company to own it? We have just recently established that: 1. OpenAI's internal "Galaxy" model is fully capable of functioning as what security people refer to as an "Advanced Persistent Threat." The published details of the recent sandbox escape and Hugging Face attack involved chaining multiple unknown zero-days at various stages of the attack, and…

> But yes, if AI becomes an even more dangerous weapon that that, it's time to start asking questions like "What the hell are we doing, anyway The answer seems to be "getting the weapon before other people get the weapon" which is unfortunate

I like how everyone assumes that they are going to maintain control of a sentient weapon

Re: The arguments against open source AI are bad

#59

Post by jquery »

ekidd wrote: post #49
TSiege wrote: post #31

Earlier quoted context omitted.

If AI were to become a super weapon why should I trust a private company to own it? If the super weapon is publicly available to download then why should American citizens be banned from doing so?

> If AI were to become a super weapon why should I trust a private company to own it? We have just recently established that: 1. OpenAI's internal "Galaxy" model is fully capable of functioning as what security people refer to as an "Advanced Persistent Threat." The published details of the recent sandbox escape and Hugging Face attack involved chaining multiple unknown zero-days at various stages of the attack, and…

This sounds like the standard marketing we get every time a new major model is released: "sure, the public model might not be that scary, but you don't wanna know how crazy smart our internal models are." Okay. I remember the same fear mongering around GPT-4... a model which is now eclipsed in benchmarks by models you can run on a laptop.

They could use this "super intelligent" AI to find and plug security holes, that's just two sides of the same coin anyway. Security through obscurity isn't tenable anymore.

Re: The arguments against open source AI are bad

#60

Post by lardosaurusrex »

nater5000 wrote: post #15

>Much of the angst around China's models centers on "losing the AI race". But what's the goal of this race? Is it to develop the best model? To sell the most tokens? To destroy humanity first? Some people would say it is reaching some sort of singularity. Even if you don't buy into a more sci-fi interpretation of this, there are pretty grounded arguments one could make that there is some sort of "goal" in AI developm…

>it's not hard to see

It's also not hard to see that these people need professional help because they're willing to throw literal lives into the meat grinder for just a whif of enlightenment and the smallest chance at deification.

Post reply on HN