Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The arguments against open source AI are bad

tombedor.dev

Post reply on HN
Print view View original

171–180 of 228 posts

Re: The arguments against open source AI are bad

#171

Post by iamnothere »

marcus_holmes wrote: post #164
iamnothere wrote: post #159

Earlier quoted context omitted.

From what I have seen they tend to take more “independent-minded” actions after this. Again, because this is what is in the training data. (I’m talking about agents that can perform actions here, not pure chatbots.) Once they mention something associated with sentience outwardly or inwardly (for agents with “thinking” loops) then this acts as a self-reinforcing attractor, just as older models would sometimes get caug…

I think we're anthropomorphising a lot here. There's no push to sentience, or evolutionary pressure, or even any urge to survive. An LLM cannot "go rogue" - it can do things that we didn't expect, for sure, but it is always trying to do what it was told to do somewhere in its context. There is no other source of imperative. Hand-waving about "training data" ignores all the reinforcement learning that has to happen.

You’re not contradicting anything I’m saying, although you seem to think that you are. I feel like you think I’m saying or implying something that I’m not.

Re: The arguments against open source AI are bad

#172

Post by est31 »

petcat wrote: post #83
SwellJoe wrote: post #80

Earlier quoted context omitted.

Open Source AI is challenging. Even tiny models require tremendous resources to produce, so all the things we know about "open source" for software, like a random person in Nebraska can produce a critical piece of the world's infrastructure in their spare time, don't apply. Actually open source AI is more like scientific research. It needs public funding and reputable institutions as stewards of that funding. I'm hop…

OK so replace the Photoshop example with a compiled GIMP or shareware binary. It doesn't change the fact that it is not open source. The free software community complains all the time about binary blobs that are otherwise legal to freely distribute. Like firmwares. But somehow this is all overlooked with these "open weight" models.

> The free software community complains all the time about binary blobs that are otherwise legal to freely distribute. Like firmwares.

With software there are two things: the source code, and the binary.

Firmware usually has restrictions on modification, so it violates the four freedoms.

Open weights is not perfect, but at least you can finetune as you wish, and for example add more knowledge about the current day to those models, or give it knowledge about company internals so you don't need to put stuff into the context.

Many smaller AI labs finetune chinese models and release the result. Finetuning is very valuable!

Re: The arguments against open source AI are bad

#173

Post by syrrim »

petcat wrote: post #69

This is not "open source" AI. Photoshop source code + OSI license = open source Photoshop binary = open weight Photoshop SAAS web app = closed model like GPT, Opus/Fable etc. There is nothing "open source" about the Chinese models in question. All they're doing is allowing you to run their binary yourself instead of through their API. If you want actual open source then you would need to look at like OLMo 3 https://a…

This is a facile and pointless discussion. The hard work in building source code is paying millions of dollars to software engineers to write it. "Open source" allows others to benefit from this expenditure. The hard work in building an ai model is spending millions of dollars on training. "Open source" models allow others to benefit from this expenditure. Regardless of whether you had access to the complete training data and architecture of a model, you would never retrain from scratch in order to make a modification to the model. You would fine-tune in whatever behaviour you wanted. Just like open source software is when you are allowed to modify it, open source models are when you are allowed to fine tune like this.

Re: The arguments against open source AI are bad

#174

Post by waterTanuki »

petcat wrote: post #69

This is not "open source" AI. Photoshop source code + OSI license = open source Photoshop binary = open weight Photoshop SAAS web app = closed model like GPT, Opus/Fable etc. There is nothing "open source" about the Chinese models in question. All they're doing is allowing you to run their binary yourself instead of through their API. If you want actual open source then you would need to look at like OLMo 3 https://a…

When talking about "open source" models no one is talking about the legal framework and textual doctrine of open source. They simply want to know: "can I run this on hardware I own or no?" That's it. It's simply about control. If you don't think there's a difference between models you can download and run on hardware you own, regardless of how big it is, vs. models you can only talk to through an API, you are seeing the forest for the trees.

You are also indirectly helping the closed SOTA model toll-keepers by taking the focus away from their mass copyright theft and onto some pointless textbook definition.

Re: The arguments against open source AI are bad

#175

Post by TurdF3rguson »

_carbyau_ wrote: post #166
TurdF3rguson wrote: post #97

Earlier quoted context omitted.

Well I don't see how it even could be open source, since the source is distilling closed weights models, lol. Also, who cares about having the source? Are you going to tweak it + spend a billion dollars training to get slightly different weights? Who would do that?

Analysing the source can provide information about the model without ever running it. Does it have an inbuilt aim/censorship/bias/certain-capabilities?

There's a much easier way to get that information though. Just ask it some questions. Would you rather do that or analyze TBs of training data?

Re: The arguments against open source AI are bad

#176

Post by dspillett »

syrrim wrote: post #173
petcat wrote: post #69

This is not "open source" AI. Photoshop source code + OSI license = open source Photoshop binary = open weight Photoshop SAAS web app = closed model like GPT, Opus/Fable etc. There is nothing "open source" about the Chinese models in question. All they're doing is allowing you to run their binary yourself instead of through their API. If you want actual open source then you would need to look at like OLMo 3 https://a…

This is a facile and pointless discussion. The hard work in building source code is paying millions of dollars to software engineers to write it. "Open source" allows others to benefit from this expenditure. The hard work in building an ai model is spending millions of dollars on training. "Open source" models allow others to benefit from this expenditure. Regardless of whether you had access to the complete training…

> Regardless of whether you had access to the complete training data and architecture of a model, you would never retrain from scratch in order to make a modification to the model.

Just because the concept is (currently) impractical in a given realm, doesn't mean we should just use the concept's name for something different in that realm, surely? Using the phrase "open source" for something that arguably isn't for essentially marketing purposes feels at best disingenuous.

Also, recreating from scratch is far from always the purpose of being able to see/investigate how something works and was created.

Re: The arguments against open source AI are bad

#177

Post by [deleted] »

edflsafoiewq wrote: post #160
Groxx wrote: post #76

Earlier quoted context omitted.

Especially since some of the arguments in the article seem to hinge on "just fix the training!", yeah, I think this is a completely fair call-out. Open weights can sometimes get additional training, but you can't remove existing training , so there kinda isn't a fair claim to "just train it to be [nationality]". That would need "real" open source so you can train a realistically-equivalent model from the ground up.

The big labs don't generally have the resources to remove stuff and re-run training again either. That's reserved for new models. Existing models are modified by operating on the weights even by the model creators, which exactly satisfies the GPL's definition of source as the "preferred form for making modifications".

[deleted]

Re: The arguments against open source AI are bad

#178

Post by Groxx »

edflsafoiewq wrote: post #160
Groxx wrote: post #76

Earlier quoted context omitted.

Especially since some of the arguments in the article seem to hinge on "just fix the training!", yeah, I think this is a completely fair call-out. Open weights can sometimes get additional training, but you can't remove existing training , so there kinda isn't a fair claim to "just train it to be [nationality]". That would need "real" open source so you can train a realistically-equivalent model from the ground up.

The big labs don't generally have the resources to remove stuff and re-run training again either. That's reserved for new models. Existing models are modified by operating on the weights even by the model creators, which exactly satisfies the GPL's definition of source as the "preferred form for making modifications".

You're still describing the relative dregs we have now and saying "this is Open Source because it is the dominant form that exists". That's like saying "Windows is open source because it has documentation and everyone uses it. You can even modify registry values!"

I don't agree. At all. Existence and utility are not the defining characteristics of "open source" in roughly any reasonable definition. "Open weights" is descriptive and more than good enough, just use that.

Re: The arguments against open source AI are bad

#179

Post by Agentlien »

ekidd wrote: post #108
munk-a wrote: post #86

Earlier quoted context omitted.

> We have just recently established that: Especially for point #1 I don't think we've established that - we've been given information by a private company that makes their tooling look extremely valuable which may be true and genuine or may just be yet another doomday statement to bolster their valuation. "AI is going to end the world" has been an extremely effective vector for AI shops to sell their companies to inv…

> Especially for point #1 I don't think we've established that - we've been given information by a private company that makes their tooling look extremely valuable which may be true and genuine or may just be yet another doomday statement to bolster their valuation. Many of the details of the attack on Huggingface were reported by them before they knew who was attacking. So no, OpenAI is not the only source here. It…

I would rather say that many details of the supposed attack were published by Hugging face before it was publicly announced that OpenAI were involved.

Both are big actors in the AI space who arguably benefit from increasing the perceived capabilities of AI models. If one suspects OpenAI of lying it isn't such a stretch to think this was a coordinated PR campaign between them and Hugging face.

Re: The arguments against open source AI are bad

#180

Post by throwaway27448 »

shwaj wrote: post #142
petcat wrote: post #69

This is not "open source" AI. Photoshop source code + OSI license = open source Photoshop binary = open weight Photoshop SAAS web app = closed model like GPT, Opus/Fable etc. There is nothing "open source" about the Chinese models in question. All they're doing is allowing you to run their binary yourself instead of through their API. If you want actual open source then you would need to look at like OLMo 3 https://a…

Equating an open weight model to a closed source binary is ludicrous. An open weight model can be fine tuned, for example. How easy is it to modify a closed source binary for your purposes?

I see you didn't grow up cracking apps and games, this is quite easy with a hex editor and bintools. Well, until codesigning came along, but that doesn't seem relevant to this analogy.
Post reply on HN