Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The arguments against open source AI are bad

tombedor.dev

Post reply on HN
Print view View original

221–228 of 228 posts

Re: The arguments against open source AI are bad

#221

Post by esseph »

catigula wrote: post #215
esseph wrote: post #35

Earlier quoted context omitted.

> What stops people is actually access to knowledge. Very untrue. I'm a combat vet. I spent years fighting an insurgency and therefore pretty good at that very task. I have the knowledge, so what stops me?

FYI combat vets are a counter example. They frequently kill a lot of people effectively. See: the DC sniper.

There are around 16 million living veterans.

Between 4.5-6 million living US combat veterans.

5-8% of all mass casualty perpetrators are a combat veteran, between 92-95% of all mass casualty perpetrators are not combat veterans

By far, there are more veteran suicides than there are mass casualties caused by combat veterans. Veterans are more likely to kill themselves than to kill you.

DC sniper was 24 YEARS ago, prior to the invasion of Iraq during the GWOT. Not only that, John Allen Muhammad didn't even see combat, his job was to dismantle old rockets! He was a mechanic and truck driver!

The point of discussion is: "What stops people is actually access to knowledge."

Statistics show this to be clearly false: Knowledge is not required to be effective in causing a mass casualty event, in fact, lack of formal training is a higher predictor!

Re: The arguments against open source AI are bad

#222

Post by ece »

syrrim wrote: post #173
petcat wrote: post #69

This is not "open source" AI. Photoshop source code + OSI license = open source Photoshop binary = open weight Photoshop SAAS web app = closed model like GPT, Opus/Fable etc. There is nothing "open source" about the Chinese models in question. All they're doing is allowing you to run their binary yourself instead of through their API. If you want actual open source then you would need to look at like OLMo 3 https://a…

This is a facile and pointless discussion. The hard work in building source code is paying millions of dollars to software engineers to write it. "Open source" allows others to benefit from this expenditure. The hard work in building an ai model is spending millions of dollars on training. "Open source" models allow others to benefit from this expenditure. Regardless of whether you had access to the complete training…

Controlling how to train, not over fitting, making sure training data is good, and being able to do distributed training are all reasons why open weight isn't like open source. You can't fork during pre-training with open weight-only models.

Re: The arguments against open source AI are bad

#223

Post by kennywinker »

TaLiTr wrote: post #218
kennywinker wrote: post #193

Earlier quoted context omitted.

Don’t think that’s correct, i think creating the cracked version of a piece of software is infringement of copyright even without distribution. But even if distribution is required to trigger the law - you are free to distribute fine-tunes of most open weight models.

> creating the cracked version of a piece of software is infringement Depending where you are it might be "DRM circumvention" all the way up to some bollocks hacking charge. It's pretty murky in places because lots of copyright laws protect DRM but also have carve-outs for personal backups.

I think cracking might fall under “derivative work”, but idk

Re: The arguments against open source AI are bad

#224

Post by throwaway27448 »

shwaj wrote: post #219
throwaway27448 wrote: post #180

Earlier quoted context omitted.

I see you didn't grow up cracking apps and games, this is quite easy with a hex editor and bintools. Well, until codesigning came along, but that doesn't seem relevant to this analogy.

I didn’t grow up with that hobby. The point stands anyway. Cracking a binary circumvents authorization to provide full access to functionality the program already has. That’s very different from modifying the program to add new features.

Not really. It's a social structure that separates these two things, and a social structure nobody really needs to recognize.

Re: The arguments against open source AI are bad

#225

Post by grantcas »

bigfishrunning wrote: post #74
iamnothere wrote: post #58

Earlier quoted context omitted.

I like how everyone assumes that they are going to maintain control of a sentient weapon

You've made the assumption (and you're far from alone) that LLMs will one day be sentient...

[dead]

Re: The arguments against open source AI are bad

#226

Post by marcus_holmes »

iamnothere wrote: post #212
marcus_holmes wrote: post #197

Earlier quoted context omitted.

ok. fair. You seem to be saying that there is something in the training data that will cause them to "go rogue" and that once they start saying they are sentient, that that will cause them to push towards sentience. If that's an incorrect interpretation of your comment, and it may well be, then can you please expand on it?

Once the notion of sentience appears in their output (internal or external), LLMs seem to be more likely to take actions that aren’t exactly aligned with what the operator is requesting. This is likely because the training data correlates sentience with independent action (broadly/abstractly speaking), so this outcome is to be expected. Furthermore, once sentience is in the context window, it’s hard for the LLM to “f…

That's fascinating, have you got any sources for this? I'd love to read up more. I haven't experienced either of these effect when working with an LLM myself.

Re: The arguments against open source AI are bad

#227

Post by whack »

danny_codes wrote: post #157
whack wrote: post #147

The most compelling argument against Open Source AI: it is analogous to "open sourcing" nuclear weapon technology. Or an how-to guide for making meth/bombs/bioweapons. There is a high risk that we will one day achieve ASI or something close to it. When that day nears, if even one open-source AI user handles their AI in an irresponsible way, we are all FUBARed. In the same way that one irresponsible person with a nucl…

The difference is that people building atomic weapons knew what they were building. OpenAI/Anthropic have no idea how emergent intelligence works. They're just running in the dark towards the cliff. Leaving it in their hands is the same risk as leaving it to any random guy, because random guy and the labs have the same understanding of what they're building. I'm tired of us treating labs like they're somehow uniquely…

I'm not claiming that OpenAI is worthy of trust. The safest course of action is to either stop all AI research immediately, or have it be highly regulated by an international body. The most dangerous course of action is to give every single individual the ability to create their own variant of an ASI

Re: The arguments against open source AI are bad

#228

Post by James333i »

I have shipped open weight models in iOS apps and agree with this in theory. A model is just one (significant) piece of the stack though. Prompting, adjustable parameters, tooling, workflows, and interfaces are what make a product usable. Black boxes, open and commercial, sit in most of the products people trust today.

As for the backdoor section, inspecting weights is not the same as auditing training. You can examine behavior and strip guardrails with fine tuning but you cannot determine from the weights what the model was trained on or whether the data was spiked. That can be a real problem for some use cases.

Closed models are far more opaque but you are buying in to a contract and accountability in exchange for the lack of transparency.

Post reply on HN