The 'arguments' are not meant to persuade, they are meant to establish and entrench an overton window (regarding 'AI' in this case).
The arguments against open source AI are bad
141–150 of 228 posts
Re: The arguments against open source AI are bad
#142This is not "open source" AI. Photoshop source code + OSI license = open source Photoshop binary = open weight Photoshop SAAS web app = closed model like GPT, Opus/Fable etc. There is nothing "open source" about the Chinese models in question. All they're doing is allowing you to run their binary yourself instead of through their API. If you want actual open source then you would need to look at like OLMo 3 https://a…
Equating an open weight model to a closed source binary is ludicrous. An open weight model can be fine tuned, for example. How easy is it to modify a closed source binary for your purposes?
Re: The arguments against open source AI are bad
#143From a neutral standpoint, governments usually have more general, comprehensive analytics, as well as greater context and insight into the risks of AI usage, and a possibly better methodology to control access to or use of open-source AI (perhaps by analyzing digital activities and consequences), because they learn from failures. On the other hand, open source creates major opportunities, and it’s hard to accept bans on something that truly benefits users acting for the common good. However, it also benefits unethical or opportunistic users—for whom regulation is indeed necessary.
Re: The arguments against open source AI are bad
#144> One probable outcome of an open-weight-model-dominant world is full AI communism...
oh no, not communism.
Re: The arguments against open source AI are bad
#145This is not "open source" AI. Photoshop source code + OSI license = open source Photoshop binary = open weight Photoshop SAAS web app = closed model like GPT, Opus/Fable etc. There is nothing "open source" about the Chinese models in question. All they're doing is allowing you to run their binary yourself instead of through their API. If you want actual open source then you would need to look at like OLMo 3 https://a…
Equating an open weight model to a closed source binary is ludicrous. An open weight model can be fine tuned, for example. How easy is it to modify a closed source binary for your purposes?
If the rapid rate at which games in the 1980s-2000s were cracked is any indication, pretty easy indeed.
Re: The arguments against open source AI are bad
#146This is not "open source" AI. Photoshop source code + OSI license = open source Photoshop binary = open weight Photoshop SAAS web app = closed model like GPT, Opus/Fable etc. There is nothing "open source" about the Chinese models in question. All they're doing is allowing you to run their binary yourself instead of through their API. If you want actual open source then you would need to look at like OLMo 3 https://a…
Equating an open weight model to a closed source binary is ludicrous. An open weight model can be fine tuned, for example. How easy is it to modify a closed source binary for your purposes?
Funny enough, it seems to be surprisingly easy with LLMs from what I've heard. :)
Re: The arguments against open source AI are bad
#147The most compelling argument against Open Source AI: it is analogous to "open sourcing" nuclear weapon technology. Or an how-to guide for making meth/bombs/bioweapons. There is a high risk that we will one day achieve ASI or something close to it. When that day nears, if even one open-source AI user handles their AI in an irresponsible way, we are all FUBARed. In the same way that one irresponsible person with a nuclear bomb can FUBAR the entire world, hence why "open sourcing nuclear weapon tech" is not even a subject of debate.
You don't have to believe that ASI is 100% likely. People like Hinton believe there's a 50% chance of existential risk from AI, but you don't have to go that far either. If there's even a tiny chance of ASI posing a threat to humanity, that's reason enough to lock it down
Re: The arguments against open source AI are bad
#148The most compelling argument against Open Source AI: it is analogous to "open sourcing" nuclear weapon technology. Or an how-to guide for making meth/bombs/bioweapons. There is a high risk that we will one day achieve ASI or something close to it. When that day nears, if even one open-source AI user handles their AI in an irresponsible way, we are all FUBARed. In the same way that one irresponsible person with a nucl…
This argumentation is going to be joked about in the future. Like downloading a car.
Would you open source a nuclear bomb???
Re: The arguments against open source AI are bad
#149Earlier quoted context omitted.
You've made the assumption (and you're far from alone) that LLMs will one day be sentient...
One interesting thing about LLMs is they sometimes seem to “assume” they are sentient and then begin acting in that mode, as the assumption of sentience influences future tokens. Whether or not this qualifies as “true” sentience is irrelevant if the effects are the same, at least from a pragmatic perspective.
They are trained and optimised to have plausible conversations. If the next plausible thing to say in the conversation is "I am sentient" then they will say that. That's not the same as actually being sentient.
Re: The arguments against open source AI are bad
#150Earlier quoted context omitted.
I tried to address safety, albeit briefly, in the sections on backdoors and propaganda. What would you have liked to see? I didn't give it much treatment because my frame of reference is about open vs closed AI, and I don't see a lot of daylight in safety concerns between the two.
The core safety-related arguments for closed weights are: 1. Visibility. API providers can monitor for misuse, and regulators only need to oversee a few centralised players. Once weights are released, anyone with sufficient hardware (foreign governments, criminal groups, terrorists?) can run them and monitoring is no longer feasible. 2. Revocability and containment. If closed-weights models are found to be dangerous,…
I trust open source code more than I trust proprietary code. And I believe history vindicates me.
Everything that you're saying can be done with open-weights models by bad people can also be done with closed-weights models by the people who own them. And there is no defence against that, we are forced to trust that the owners are not bad people. Those owners have proven time and again that they are only interested in profit and/or control. They will do whatever they think makes them the most money, or gives them the most control. They will also lie about this.
At least with open weights we know the danger and can do something about it. With closed weights we don't know what's going on.